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Introduction 



State law requires the California 
Postsecondary Education Commission to 
review and comment on the need for new 
degree and certificate programs proposed 
by the public higher education systems. 

This report contains three policy 
recommendations intended to increase the 
likelihood that current and proposed 
doctoral programs in educational 
leadership will achieve desired results and 
consequences. Recommendations for 
enhancing leadership proposals are also 



provided. 
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The Commission advises the Governor and the 
Legislature on higher education policy and fiscal 
issues. Its primary focus is to ensure that the 
State’s educational resources are used effectively 
to provide Californians with postsecondary educa- 
tion opportunities. More information about the 
Commission is available at www.cpec.ca.gov . 
Commission Report 07-25 



Legislation passed in 2005 authorizes the California 
State University (CSU) to award the Doctorate of 
Educational Leadership independent of the Univer- 
sity of California (UC). Following a comprehensive 
review of eight CSU proposals for new doctoral 
programs in educational leadership, the California 
Postsecondary Education Commission recommends 
that: 

1. The California State University, the University 
of California, and the Association of Independ- 
ent California Colleges and Universities 
(AICCU) develop collaboratively a strategic 
plan for preparing educational leaders in an era 
marked by high State and national expectations 
for public schools and community colleges. The 
central purpose of statewide and regional plan- 
ning would be to prepare educational leaders to 
initiate K-14 reform efforts that will lead to de- 
monstrative improvements in the quality of 
teaching, instruction, and student academic 
achievement. 

2. The State invest program funds to help support 
the development of evaluative tools that could 
be used at an appropriate time in the future to 
assess the collective effects of public and pri- 
vate/independent doctoral leadership programs 
on K-14 student learning and reform. 

3. The State encourage public and pri- 
vate/independent universities to include K-14 
performance measures in their evaluation plans 
to help inform assessment of the collective re- 
gional and statewide effects of doctoral 
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leadership programs on student learning and achievement. Such measures might include student profi- 
ciency scores, California High School Exit Exam (CAHSEE) results, college-going rates, community 
college transfer rates, and school Academic Performance Index (API) scores. 



The Rationale for Doctoral Leadership Programs 

The level of student academic achievement at any point in time is a direct consequence of many factors, 
including attainment of perquisite content knowledge and skills (student readiness), student motivation, 
quality of teaching and instruction, teacher training, and assessment practices that precisely pinpoint 
teaching and learning deficiencies. Student achievement is also a consequence of a combination of sup- 
port factors, including the adequacy of fiscal and learning resources; school management practices; vi- 
sionary school leadership; parental support; academic preparation and equity programs; school safety; 
and collaborative alliances that consist of the K-12 system, the higher education system, California’s 
industry and business community, and philanthropic partners. 

School administrators and school management practices, in particular, have been the subject of exten- 
sive debate at the local, state, and federal level. It is now readily acknowledged that state and federal 
standards-based reform practices are requiring school administrators to manage public schools much dif- 
ferently than in the past and to be keenly attentive to a wider range of public expectations. Arthur Levine 
describes these challenges in his report, Educating School Leaders (2005). The excerpt shown below is 
from that report. 

In an outcome-based and accountability driven era, administrators have to lead their schools in 
the rethinking of goals, priorities, finances, staffing, curriculum, pedagogies, learning resources, 
assessment methods, technology, and use of time and space. They have to recruit and retain top 
staff members and educate newcomers and veterans alike to understand and become comfortable 
with an education system undergoing dramatic and continuing change. They have to ensure the 
professional development that teachers and administrators need to be effective. They have to 
prepare parents and students for the new realities and provide them with the support necessary 
to succeed. They have to engage in continuous evaluation and school improvement, create a 
sense of community, and build morale in a time of transformation. 

In adopting Chapter 269 of the California Education Code , Statutes of 2005, the California Legislature 
formally acknowledged the crucial role that school leaders assume in promoting effective school and 
student success. The statute authorizes the California State University to offer doctoral programs in 
educational leadership to prepare leaders and administrators to participate effectively in school reform 
efforts and to formulate administrative practices that will lead to improvements in K-14 instruction and 
learning. The statute also requires the CSU to design and operate doctoral leadership programs in part- 
nership with public schools and community colleges. Previously, the State University had been limited 
to offering the doctor of education degree only in partnership (jointly) with the University of California, 
or in partnership with one of the state’s independent colleges and universities. 

Section 66040.3 of the Education Code expresses the intent of the Legislature that the California Post- 
secondary Education Commission review and comment on all CSU educational doctoral proposals to 
ensure, among other important considerations, that such proposals address specific educational leader- 
ship needs. The following excerpt references the Commission’s responsibility with respect to CSU doc- 
toral leadership proposals. As stated: 

Nothing in this article shall be construed to limit or preclude the California Postsecondary Edu- 
cation Commission from exercising its authority under Chapter 11 — commencing with Section 
66900 — to review, evaluate, and make recommendations relating to any and all programs estab- 
lished under this article. 
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Although CSU leadership proposals are being tailored to address specific local and regional challenges, 
it appears that all campuses share a common mission to provide doctoral-intensive instruction and clini- 
cal research experiences that will assist practitioners in (a) identifying and implementing best practices, 
(b) managing schools in a more fiscally- sound manner, and (c) promoting equitable educational oppor- 
tunities. The CSU Chancellor’s Office has requested that each campus use California Professional Stan- 
dards for Educational Leaders, as well as examining other educational leadership programs in the coun- 
try, as a framework in developing curricula and in crafting the overall design of proposed leadership 
programs. State- supported leadership programs have the potential to positively impact schooling and 
student learning if: 

1. The programs are successful in attracting and enrolling educational leaders that are currently work- 
ing in the field, including district and county superintendents, school principals, educational re- 
searchers and analysts, and community college presidents and department chairpersons. 

2. The leadership programs are aligned with K-14 leadership needs. 

3. The programs are of the highest quality. 

4. Appropriate evaluative tools are developed to assess the impact of leadership programs on school 
and community college success. 



Policy Recommendations 

This section contains three policy recommendations intended to increase the likelihood that public and 
private/independent doctoral programs in educational leadership will have desired instructional out- 
comes and societal benefits. 

Policy Recommendation Regarding Collaboration 

The California State University, the University of California, and the Association of Independent Cali- 
fornia Colleges and Universities should develop collaboratively a strategic plan for preparing educa- 
tional leaders in an era marked by high state and national expectations for public schools and commu- 
nity colleges. The central purpose of statewide and regional planning would be to prepare educational 
leaders to initiate K-14 reform efforts that will lead to demonstrative improvements in the quality of 
teaching, instruction, and student academic achievement. In developing the plan, it is recommended that 
the higher education systems consult with the State’s P-16 Council. 

Policy Rationale 

As shown by Display 1, eight University of California campuses and 17 private and independent univer- 
sities offer doctoral concentrations in education. All but one of these institutions offer concentrations in 
leadership. Of the eight UC leadership programs, four are offered jointly with the State University. The 
CSU’s long-range plans indicate a desire for each of its comprehensive campuses to offer the Doctorate 
in Educational Leadership independently, while the CSU system will continue to support the four joint 
programs that have been established. If the long-range plans are approved and implemented, the number 
of public and private institutions awarding doctorates in educational leadership would total 46. Pres- 
ently, the State has no readily available method of determining the extent to which the combination of 
public and private/independent doctoral programs are meeting K-14 statewide and regional leadership 
needs in an efficient, cost-effective manner. 

It is important to note that many of the leadership programs shown in Display 1 offer a common founda- 
tional curriculum to meet leadership needs, while also offering areas of specialization. For example, a 
doctoral program that emphasizes transformational leadership might focus on effective methods that 
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leaders can employ to increase the capacity of schools and school districts to innovate and support 
change. A program in organizational leadership might emphasize the various political, structural, sym- 
bolic, and human resource frameworks for understanding the role of leadership in organizations, whe- 
reas instructional leadership programs might place greater emphasis on effective teaching and instruc- 
tional practices. 



DISPLAY 1 Public and Private California Universities Offering Doctoral Concentrations 
in Education 



University of California 


Main Campus Location 


Education Leadership Fields 


UC Berkeley 


Berkeley 


Ed.D Educational Leadership 


UC Davis (Jointly w / CSU) 


Davis 


Ed.D Educational Leadership 


UC Irvine 


Irvine 


Ed.D Administration and Leadership 


UC Los Angeles 


Los Angeles 


Ed.D Educational Leadership 


UC Riverside 


Riverside 


Ph.D Leadership and Policy Studies 


UC San Diego (Jointly w/ CSU) 


La Jolla 


Ed.D Educational Leadership 


UC Santa Barbara (Jointly w / CSU) 


Santa Barbara 


Ed.D Educational Leadership 


UC Santa Cruz (Jointly w / CSU) 


Santa Cruz 


Ed.D Educational Leadership 


WASC-Accredited Non-public 4-Year 
Institutions 


Main Campus Location 


Education Leadership Fields 


Alliant International University - San Diego 


San Diego 


Ed.D Leadership and Management 


Azusa Pacific University 


Azusa 


Ed.D Educational Leadership 


California Lutheran University 


Thousand Oaks 


Ed.D Educational Leadership 


Chapman University 


Orange 


No Leadership Concentration 


Claremont Graduate University 


Claremont 


Ed.D Administration and Leadership 


Fielding Graduate University 


Santa Barbara 


Ed.D Educational Leadership 


La Sierra University 


Riverside 


Ed.D Educational Leadership 


Loyola Marymount University 


Los Angeles 


Ed.D Educational Leadership 


Mills College 


Oakland 


Ed.D Educational Leadership 


Pepperdine University 


Malibu 


Ed.D Educational Leadership 


Stanford University 


Stanford 


Ph.D Educational Administration 


University of La Verne 


La Verne 


Ed.D Organization Leadership 


University of San Diego 


San Diego 


Ed Educational Leadership 


University of San Francisco 


San Francisco 


Ed.D Organization and Leadership 


University of Southern California 


Los Angeles 


Ph.D K-12 Leadership 


University of Redlands 


Redlands 


Ed.D Educational Leadership and Justice 


University of the Pacific 


Stockton 


Ed.D Administration and Leadership 



It is also important to note that key distinctions exist between the professionally-oriented Ed.D degree 
and the research-oriented Ph.D degree. There is, however, an appreciable degree of overlap among vari- 
ous specialty leadership programs and between the Ed.D and Ph.D degree requirements in education. 
For example, the school leadership curriculum proposed by CSU campuses would require doctoral can- 
didates to (a) pass a qualifying examination, (b) successfully prepare and defend a doctoral dissertation, 
and (c) successfully complete coursework in program evaluation and assessment, strategic planning, cur- 
riculum and school reform, educational leadership, resource and fiscal planning, data-driven decision- 
making, theories of cross-cultural education, and qualitative and quantitative research methods. Many 
Ph.D programs in education administration and leadership consist of a similar foundational curriculum. 
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Regional planning would be one effective way to identify the common and unique features of leadership 
programs, and how such programs collectively can support K-14 leadership needs. 

Here is a regional planning example involving the Los Angeles County Region. Notice from Display 2 
that there are currently six doctoral leadership programs serving the region. Display 3 shows what this 
region would look like spatially if the State were to fund new leadership programs at CSU Northridge, 
Los Angeles, Dominguez Hills, and Long Beach. This would increase the number of leadership pro- 
grams serving the region from six to 10. What would be the value-added of increasing doctoral leader- 
ship production in the Los Angeles County Region? This question could be addressed a priori through 
effective regional planning. 

DISPLAY 2 Regional Map of California, and campuses in Los Angeles County Region 
Offering Educational Leadership Doctoral Programs 
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DISPLAY 3 Los Angeles County Region: Current and Prospective Locations of Educational 
Leadership Doctoral Programs 




Note: Yellow squares are existing programs; stars represent prospective programs. 



Effective regional planning would also enable the public and independent higher education systems to: 

• Conduct credible supply-demand analyses so that the State would have a more informed under- 
standing of the need for additional school leadership programs on a regional basis. 

• Identify the common and unique features of leadership programs serving a region and how those 
features address local K-14 leadership needs. 

• Identify features of various Ph.D. programs in education serving a region that address specific 
local K-14 leadership needs. 

• Help prospective doctoral applicants select from among the Ed.D and Ph.D programs serving a 
region the one that best matches their specific doctoral training needs. 

• Develop effective ways to increase the pool of school leaders from ethnically-underrepresented 
backgrounds and from economically-disadvantaged backgrounds. 



Policy Recommendation Regarding Public Investment in the Evaluation of Doctoral 
Programs in Educational Leadership Related Fields 

The State should invest program funds to help support the development of evaluative tools that could be 
used at an appropriate time in the future to assess the collective effects of public and private/independent 
doctoral leadership programs on K-14 student learning and reform. 
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Policy Rationale 

On a national level, the literature reviewed for this report noted the difficulty in discerning the impact of 
educational leadership programs because little effort has been directed by universities across the nation 
to produce evidence that informs practitioners, scholars, and policymakers on the effectiveness of such 
programs. State law requires the CSU, in collaboration with the Department of Finance and the Legisla- 
tive Analyst’s Office, to conduct a statewide evaluation of its doctoral leadership programs by January 
1, 2011. The evaluation is to include: (a) an assessment of the extent to which the programs are fulfill- 
ing educational leadership training needs, based in part on a supply-demand analysis; and (b) the collec- 
tion of available evidence on the extent to which leadership programs are enhancing K-14 student learn- 
ing and reform. 

The Commission believes that a useful evaluation will require a mixed-methods approach that relies on 
both quantitative (quasi-experimental, descriptive) and qualitative (case studies, personal interviews) 
design features. Considerable time and expertise will be required for development and field testing. Al- 
though evaluation expertise resides within the CSU, the Department of Finance, and the Legislative 
Analyst’s Office, the development of assessment tools will require external consultants who have exten- 
sive experience in program evaluation. In the absence of State General Fund support for the evaluation 
process, it is unlikely that in-depth assessments will be produced. 

The Commission wishes to emphasize that it is not calling for the type of evaluations described here to 
be conducted in the near term. Strategic statewide and regional plans must be developed first, and the 
higher education systems must be given a reasonable amount of time to fine-tune their leadership pro- 
grams. The State must also be willing to invest in the development and pilot-testing of evaluative tools. 
The earliest that such evaluations could be undertaken would be about the middle of the next decade. 
But the time to start systematic evaluation planning is now. 

Policy Recommendation Regarding K-14 Student Performance Data 

The Commission recommends that the CSU and the UC include K-14 performance measures to help 
inform assessment of the collective effect of doctoral leadership programs on student learning and 
achievement. Such measures might include student proficiency scores, California High School Exit Ex- 
amination (CAHSEE) results, college-going rates, community college transfer rates, and school Aca- 
demic Performance Index (API) scores. Performance results should be reported on both a statewide and 
regional basis. 

Policy Rationale 

A major criticism of doctoral leadership programs that is noted in the literature is that university self- 
assessments have focused almost exclusively on basic institutional measures and less on long-term im- 
pact of those programs on student achievement. Basic institutional measures include information on 
student admits, student enrollments, ethnic-racial composition, degrees earned, and the amount of re- 
sources required to produce a doctoral graduate. Little effort has been directed towards assessing envi- 
ronmental changes that have resulted as a consequence of leadership programs. 

California adopted various K-12 accountability performance measures that have been subjected to rig- 
orous reliability and validity standards. Those measures include student proficiency scores in mathemat- 
ics, language arts, social sciences, and the physical and biological sciences; CAHSEE results; and school 
API scores. Because an ultimate aim of educational leadership programs is to enhance K-12 schooling, 
the Commission strongly believes that there should be some connection between advanced doctoral 
study undertaken by school leaders and their subsequent influence on school management practices and 
student performance. 
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The reluctance to embrace school performance measures in the evaluation of doctoral leaderships pro- 
grams is associated with two common concerns. First, some education faculty members argue that ef- 
fective schooling is associated with a host of cognitive and institutional factors. As such, it is not possi- 
ble to separate the effects of doctoral training from other improvement factors. Second, some university 
faculty and department chairs do not want the success of their leadership programs to rest on the rate of 
improvement in student performance. 

The Commission agrees that effective schooling is indeed associated with a wide range of factors (see 
page 2 of this report for examples). Given the number of school leaders that higher education systems 
anticipate enrolling on an annual basis in the near future (430 or more), the State must have a reasonable 
understanding of how its investment in school leadership programs is contributing to student success. 
Not to do so would likely compromise the public trust in doctoral training. Appendix A contains an an- 
notated bibliography of research studies examining the indirect effect of school leadership on school re- 
form and student academic achievement. 

Recent National Research Regarding School Leadership Programs: 
Lessons to be Learned 

Despite the challenges confronting school leaders and administrators, the potential for school leadership 
programs to have a positive impact can be maximized when the four critical factors listed on page 3 are 
addressed. High-quality leadership programs are of little value unless practitioners who enroll in these 
programs sincerely want to make a difference, as opposed to simply obtaining an advanced degree in 
order to increase salary and related executive compensation benefits. 

Recent scholarly research and literature on various doctoral leadership programs offer keen insights. 
Levine’s four-year study (2005) involved a national representative sample of 28 education graduate 
schools. These schools were selected to reflect the diversity of the nation’s education schools with re- 
spect to region, race, gender, religion, and Carnegie classification. Data were obtained from deans, de- 
partment chairpersons, education school faculty members and alumni, and school principals. Education 
departments and schools in the sample were rated in relation to the following nine criteria: 

• Program purpose 

• Curricular coherence 

• Curricular balance 

• Faculty composition 

• Admission selectivity 

• Degree requirements 

• Scholarly research 

• Financial resources 

• Assessments 

The authors generally found the overall quality of educational administration programs to be poor, even 
for some of the nation’s leading universities. Exceptions included the Education School at the Univer- 
sity of Wisconsin, Madison, and the Peabody College of Vanderbilt University (Levine, 2005, page 23). 
Common criticisms were that school leadership programs do not engage in systematic self-assessment; 
curricula are disconnected from the needs of leaders and their schools; the professoriate are ill-equipped 
to educate school leaders; faculty research is detached from practice; and programs generally receive 
insufficient funding. 
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The 2002 edition of Leadership and Public Policy in Schools includes an article that also underscored 
the need for better assessment practices. The authors argued that it is difficult to discern the impact of 
educational leadership programs. Universities have directed little effort in producing credible evidence 
that informs practitioners, scholars and policymakers on such programs’ effectiveness. The good news, 
according to the researchers, is that many universities are reforming their leadership programs in mean- 
ingful ways, such as: (a) using cohort groups, (b) aligning courses with professional standards, and (c) 
strengthening field experiences. The bad news is that universities have shown little interest in collecting 
data to link reform efforts to the asserted purpose of producing capable leaders who can enhance student 
learning and that can better prepare our nation’s youth for jobs and citizenship. 

The authors of a WestED article, Turning Around Low Performing Schools and Districts (2007), ob- 
served that many school administrators fail to use performance data to find the right focus for school 
reform, and that they often seek a prescription before studying the disease. 



Enhancing California Educational Leadership Proposals 

There is universal agreement that quality programs start with quality proposals. That is, when a proposal 
establishes a compelling case based on supporting empirical evidence of need, it is more likely that the 
intended program will be of high quality and will achieve its desired outcomes and societal benefits. 

Findings Regarding the Most Recent CSU Educational Leadership Proposal 

Commission staff found the most recent CSU educational leadership proposal to have been developed 
exceptionally well. Staff was especially appreciative of the inclusion of empirical data to support re- 
gional needs. The proposal information enabled staff to conduct a review in a timely manner. The only 
substantive staff comment was that the host campus should consider consulting with its Department of 
Economics to develop a more refined labor market supply-demand analysis. 

Findings Regarding the Initial Set of CSU Leadership Proposals 

A number of concerns and recommendations were conveyed to the State University regarding the initial 
set of leadership proposals reviewed by the Commission. The concerns, outlined below, were summa- 
rized in the Commission’s 2007 Annual Program Review Report, and have since been discussed with a 
wide range of policy research agencies, including the Western Association of Schools and Colleges 
(WASC), the Legislative Analyst’s Office, the Department of Finance, and WestED. The Commission 
is pleased that the most recent CSU leadership proposals contained practically no weaknesses. 

Initial Concerns Regarding the Connection between Intended Outcomes and Workforce 
and Knowledge Needs 

The Commission’s guidelines require that proposals contain a reasonably informed description of the 
state’s workforce and knowledge needs that would be addressed by a new degree program. Most of the 
initial proposals reviewed by the Commission made general reference to the State’s educational leader- 
ship needs, but were not written with the level of specificity required by the Commission’s guidelines. 
For example, one CSU campus proposal stated: 

The gap between the achievement of students of color and low socioeconomic status and that of 
their privileged counterparts continues to plague California PreK-12 schools, despite some 
gains associated with the standards-based reform efforts. The state’s community colleges con- 
tinue to struggle to improve their transfer rates to an acceptable level. The struggles of these 
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two segments of the California education system seri’e to limit the opportunities for educational, 
social, and economic and societal well-being of the State of California. 

The rationale did not include specific school achievement measures or statistical information that de- 
scribed the magnitude of leadership challenges related to student academic achievement and community 
college transfer. Further, the rationale did not reference a level of community college transfer the fac- 
ulty at the CSU campus would consider acceptable for its particular region. The case for CSU educa- 
tional doctorate proposals could be made more compelling through the inclusion of statistical evidence 
quantifying the challenges confronting school and community college administrators and how adminis- 
trators with advanced leadership skills might more effectively manage public schools and colleges and 
lead district-level reform efforts to improve student-learning outcomes. Because the CSU is a regional 
system, it would be more informative if evidence of need were region- specific. 

As an example, a particular region of the state could be marked by: (a) high teacher turnover; (b) low 
student performance; (c) significant student attrition; (d) low college-going rates; (e) inadequate class- 
room and laboratory facilities; (f) significant numbers of English-language learners; (g) a high propor- 
tion of district teachers who are not fully credentialed; and (h) chronic unemployment among certain 
ethnic -racial groups. 

What specific skills and domain knowledge at the doctoral level would assist administrators and superin- 
tendents in effectively managing schools with such challenges? The first task is to more clearly define 
challenges by reporting statistical evidence. For instance, regional API scores could be used as a meas- 
ure of school performance. Historical rates of UC and CSU freshman participation could be used to as- 
sess college- and university-going rates. Dropout rates and CAHSEE performance by ethnicity and gen- 
der could be used to assess regional student attrition. 

The Commission’s intent is not to prescribe the full range of evidentiary information to be included in 
proposals. Rather, the interest is to assist State University leaders in thinking critically about what sta- 
tistical evidence will be most helpful in establishing their case that an urgent and compelling need for 
educational leadership doctoral programs exists. Including such statistical evidence in proposals will 
help focus subsequent evaluations of program effectiveness. 

Concerns Regarding Societal Need — Workforce Demand and Supply Component 

The Commission recognizes that the CSU intends to better prepare a preexisting administrative work- 
force by attracting and enrolling qualified individuals who are already employed in school and commu- 
nity college leadership positions. However, because the CSU also intends to prepare and train aspiring 
educational leaders, a comprehensive supply and demand analysis is required. 

Display 4 illustrates one way to array regional supply and demand data elements. These elements are 
related to the absolute size of the school administrative workforce; the annual number of administrative 
hires; the annual number of workforce separations and leaves; educational leadership doctoral and mas- 
ter’s degree production; and public community college and K-12 enrollment. These elements are high- 
lighted because the opportunity for the CSU to positively impact the management of public schools in 
any particular region is tied to them. 
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DISPLAY 4 Illustrative Example of a Supply-Demand Table for Educational 
Administrators 





2001 


2002 


2003 


2004 


2005 


2006 


Public School Administrative Workforce 

(Superintendents of any level. 

Central Office Administrators, 

Principals, Assistant Principals) 

Annual Workforce Separations 

Public School Administrative Hires 

(Superintendents of any level, 

Central Office Administrators, 

Principals, Assistant Principals ) 

Doctoral Leadership Degree Production 
of Public and Private Institutions Serving the 
Region 

Master’s and Doctoral Degrees in Other Rele- 
vant Fields of Education Award by Public and 
Private Institutions Serving the Region 

Public K-12 Regional Enrollment Data 

Community College Regional Enrollment 
Data 















Industry supply and demand data could have a number of helpful uses. The current ratio of regional 
workforce data to public school enrollment could be applied to the Department of Finance’s 10-year en- 
rollment projections in a given region to help estimate the annual number of new school administrative 
hires necessary to keep pace with public school enrollment growth. Estimates of doctoral degree pro- 
duction in educational leadership could be cross-tabulated with estimates of new administrative hires 
and help the CSU determine the relative opportunity it has to impact the skill and knowledge composi- 
tion of a given regional administrative workforce. 

Commission Concerns Regarding the Program Evaluation Component 

In considering evaluation plans associated with proposals for new degree programs, the Commission 
seeks to ensure that plans are reasonable and sound, and that they respond to any concerns expressed in 
legislation. In this regard, Section 66040.7(b) of the California Education Code calls for the State Uni- 
versity to develop education plans that involve, among other considerations, the collection and assess- 
ment of “available evidence on the effects that graduates of the programs are having on elementary and 
secondary school and community college reform efforts and on student achievement.” The Commis- 
sion’s interest is in knowing that reasonable formative steps are being taken to ensure that appropriate 
data will be collected at a reasonable time in the future to yield useful and meaningful results, as in- 
tended in legislation. 

The Commission believes it is possible and appropriate for the State University, given Section 
66040.7(b) of the California Education Code, to consider how an evaluation plan could be used to assess 
the indirect effect that doctoral leadership programs are intended to have on school reform and student 
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academic achievement. Such an evaluation procedure should be based on some theoretical model that 
connects management practices to schooling. It is readily acknowledged that theoretical models support 
the practice of evaluation by helping researchers to ask the right questions, to organize findings, and to 
provide insights into why one would expect various factors to be causally related. 

Display 5 is a theoretical representation of direct and indirect influences on student learning. In this hy- 
pothetical example, classroom teaching and student preparedness have a direct influence on student 
learning. That is, the arrows from those two factors connect directly to student learning. The joint doc- 
toral program is intended, among other outcomes, to significantly enhance the leadership and decision- 
making skills of school administrators by providing them with critical and practical understanding of a 
planning tool called evidence-based decision-making. Evidence-based decision-making is hypothesized 
to lead to improved school management practices. This, in turn, leads to enhanced teacher development 
opportunities. If all the mediating factors were positively impacted, then learning would be expected to 
improve. 



DISPLAY 5 Hypothetical Path Analysis Depicting the Indirect Effect of a Doctoral Educational 
Leadership Program on Student Learning 




The Commission believes that such a path analysis could serve three vital purposes with respect to pro- 
gram review. First, and foremost, it would help program developers to think in a more exacting manner 
about the path by which a doctoral program in educational leadership could influence K-14 student 
learning. Second, the path analysis is likely to help program developers to be more attentive to key me- 
diators that impact both the program and the terminal outcome of enhanced student learning. Third, the 
analysis could help developers decide when assessments should be undertaken. From Display 5, it 
seems that the State would want to know at a minimum if the practice of evidence-based decision- 
making is enhancing the practice of school management and planning. A number of evaluative designs 
could be used to help determine the extent to which the implementation of best practices on the part of 
superintendents and school principals who received CSU doctoral training are enhancing student learn- 
ing in critical knowledge domains. Such evaluative methods include quasi-experimental, descriptive, 
and case- study designs. 

There is significant support among the research community for using a path analysis approach for as- 
sessing the indirect effect of leadership on student academic achievement. In a major review of studies 
conducted during the 15-year period from 1980 to 1995, Hallinger and Heck (1998, page 167) con- 
cluded, after considering the range of statistical methods used in those studies, that “mediated-effects 
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studies (path analysis) offer concrete indications of possible means through which leadership may 
achieve an impact on the school’s outcomes and effectiveness.” 

Commission Concerns Regarding Program Costs 

Some CSU doctoral proposals have not used a consistent method for deriving Full-Time Equivalent Stu- 
dents (FTES) calculations. FTES is a key determinant of anticipated instructional cost and State mar- 
ginal costs funding. For graduate programs, the Department of Finance and the CSU Chancellor’s Of- 
fice defines the term Annual Full-time Equivalent Students as the annual number of graduate credit units 
divided by 24. Fall term FTES is defined as the number of graduate credit units divided by 12. Com- 
mission staff, therefore, recommended that revenue tables include: (1) projected student credit units for 
the academic year, including summer session; (2) projected FTES, derived by dividing credit units by 
24; and (3) marginal cost revenue, derived by multiplying the marginal cost rate by FTES. 

Another concern is that some CSU doctoral proposals did not clearly report all anticipated program costs 
and revenues. The Commission’s program review guidelines require institutions to identify the fund 
sources necessary for a campus to offer a new program in the near-term and in the long run. Commis- 
sion staff requested that future leadership programs contain a five-year resource table that shows total 
anticipated program costs by expenditure category, and anticipated revenues by funding source. Fund- 
ing sources are to include projected State FTES marginal costs funds, doctoral student fee revenue, pro- 
fessional fee revenue, financial aid set-aside funds, capital outlay funds, and funds from private sources. 
Expenditure categories are to include administrative, instructional, library, student support services, and 
capital and maintenance expenses. 



Examining the K- 1 2 Challenge 

Public School Accountability 

California’s school accountability and reporting system is intended to raise the academic achievement of 
all publicly enrolled students by establishing rigorous proficiency content standards and by monitoring 
results and progress. Each year, students attending primary and secondary public schools in California 
are required to take California Standardized Tests (CST) in English/language arts, mathematics, social 
sciences, and science. These tests are aligned with state-adopted content standards describing the 
knowledge and skills that students are expected to be taught and to master at each grade level. 

Performance results in various subject areas are weighted differentially to derive a school performance 
measure called the Academic Performance Index (API). Growth targets, referred to as Adequate Yearly 
Progress, are established for public elementary and middle schools and districts; high schools and high 
school districts; unified school districts, and county offices of education. 

Progress in Student Performance 

Recent student performance data indicate that although student outcomes are far below federal target 
levels established for year 2013-14, California is achieving and sustaining a degree of progress in en- 
hancing student learning. Since 2003, when all California tests were aligned to common performance 
standards, the percentage of public students in grades 2 through 11 who scored proficient in Eng- 
lish/language arts increased from 35% in 2003 to 43% in 2007. During the same three-year period, pro- 
ficiency scores in mathematics increased from 35% to 41%. Percentage point change results are high- 
lighted in the last column of Displays 6 and 7, respectively. 
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DISPLAY 6 English/Language Arts, 2003-2007 

Percentage of Students Scoring at and Above Proficient 



Grade 


2003 


2004 


2005 


2006 


2007 


Change in 
Percentage 
2006-2007 


Change in 
Percentage 
2003-2007 


2 


36 


35 


42 


47 


48 


1 


12 


3 


33 


30 


31 


36 


37 


1 


4 


4 


39 


39 


47 


49 


51 


2 


12 


5 


36 


40 


43 


43 


44 


1 


8 


6 


36 


36 


38 


41 


42 


1 


6 


7 


36 


36 


43 


43 


46 


3 


10 


8 


31 


33 


39 


41 


41 


0 


10 


9 


38 


37 


43 


43 


47 


4 


9 


10 


33 


35 


36 


37 


37 


0 


4 


11 


32 


32 


36 


36 


37 


1 


5 


State Total 2-11 


35 


35 


40 


42 


43 


1 


8 



DISPLAY 7 Mathematics, 2003-2007 

Percentage of Students Scoring at and Above Proficient 





2003 


2004 


2005 


2006 


2007 


Change in 
Percentage 
2006-2007 


Change in 
Percentage 
2003-2007 


Grade 2 


53 


51 


56 


58 


59 


1 


6 


Grade 3 


46 


48 


54 


57 


58 


1 


12 


Grade 4 


45 


45 


50 


54 


57 


3 


12 


Grade 5 


35 


38 


44 


48 


49 


1 


14 


Grade 6 


34 


35 


40 


42 


42 


0 


8 


Grade 7 


30 


33 


37 


41 


40 


-1 


10 


General Mathematics 


20 


20 


22 


22 


21 


-1 


1 


Algebra I 


21 


18 


19 


23 


23 


0 


2 


First time test takers 










26 






Repeat test takers 










15 






Geometry 


26 


24 


26 


26 


24 


-2 


-2 


Algebra II 


29 


24 


26 


25 


27 


2 


-2 


Summative High School Math 


43 


41 


45 


46 


48 


2 


5 


Integrated 1 


r 7 


1 7 


r 7 


1 9 


1 9 


0 


2 


State Total, Grades 2-7 
and End-of-Course tests 


35 


34 


38 


41 


41 


0 


6 
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While the gains in language arts and mathematics are encouraging, it is well-understood by policymak- 
ers and educators that unless the rate of progress in student academic preparation and achievement is 
accelerated, the State will face a host of social and economic problems in the future, including: 

• A shortage of young adults with the intellectual capacity to fill entry-level positions in industries and 
businesses that are vital to California’s economic health; 

• A decline in the proportion of high school graduates who are fully prepared for the academic de- 
mands of postsecondary education; 

• An insufficient number of students in the educational pipeline available to fill vital professional oc- 
cupations for which a baccalaureate degree is required; and 

• An insufficient number of students in the educational pipeline with the cognitive and creative capac- 
ity to pursue graduate-level instruction in science-based fields that will enable California to maintain 
its competitive edge in the realm of scientific discovery and technological innovation. 

Avoidance of this last potential problem might be especially challenging. As shown in Display 8, the 
level of student proficiency in any of the science subject matter areas during the 2007 reporting period 
did not exceed 37%. 



DISPLAY 8 Science, End-of-Course, 2003-2007 

Percentage of Students Scoring at and Above Proficient 



Test 


2003 


2004 


2005 


2006 


2007 


Change in 
Percentage 
2006-2007 


Change in 
Percentage 
2003-2007 


Earth Science 


21 


22 


23 


23 


26 


3 


5 


Biology 


37 


30 


32 


35 


37 


2 


0 


Chemistry 


31 


28 


27 


27 


31 


4 


0 


Physics 


29 


29 


31 


32 


36 


4 


7 


Integrated 1 


7 


5 


8 


9 


11 


2 


4 


Integrated 2 


8 


8 


6 


5 


7 


2 


-1 


State Total, 
End-of-Course Tests 


29 


24 


27 


28 


31 


3 


2 



Source: California Department of Education. 



Student Performance by Gender and Selected Ethnic-racial Groups 

Displays 9 and 10 reveal that student proficiency varies by gender and ethnic -racial group. In 2007, fe- 
males outperformed males in language arts proficiency by eight percentage points. The performances of 
males and females in mathematics proficiency were nearly the same, with males slightly ahead of fe- 
males by a single percentage point. 

Differences in proficiency by ethnic -racial groups are more noticeable. In 2007, 66% of Asian students, 
and 62% of White students were proficient in language arts. Thirty-one percent of African Americans 
and 29% of Latino students tested at or above proficiency in 2007. Both ethnic-racial groups have made 
significant progress since 2003, when 22% of African Americans and 20% of Latino students were pro- 
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ficient in language arts. Superintendent of Public Instruction Jack O’Connell has stated that the ethnic- 
racial achievement gap is unacceptable, and that intense intervention efforts are at work in low- 
performing schools with significant numbers of African American, Latino, and socio-economically dis- 
advantaged students. 

In terms of mathematics 67% of Asian students and 53% of White students scored at or above profi- 
ciency. Twenty-five percent of African American, and 30% of Latino students tested proficient. 



DISPLAY 9 Language Arts Proficiency Results by Gender and Selected Ethnic-racial Groups 



Ethnic-Racial Group/Gender 


2003 


2004 


2005 


2006 


2007 


Change 


African American 


22% 


23% 


27% 


29% 


31% 


8 


Asian (excludes Filipino students) 


55% 


56% 


62% 


64% 


66% 


11 


Hispanic/Latino 


20% 


21% 


25% 


27% 


29% 


9 


White 


53% 


54% 


58% 


60% 


62% 


9 


Female 


39% 


40% 


44% 


46% 


48% 


9 


Male 


31% 


32% 


36% 


38% 


40% 


9 



Source: California Department of Education. 



DISPLAY 10 Mathematics Proficiency Results by Gender and Selected Ethnic-racial Groups 



Ethnic-Racial Group/Gender 


2003 


2004 


2005 


2006 


2007 


Change 


African American 


19% 


19% 


23% 


24% 


25 


6 


Asian (excludes Filipino students) 


60% 


60% 


65% 


67% 


67 


7 


Hispanic/Latino 


23% 


23% 


27% 


30% 


30 


7 


White 


47% 


46% 


51% 


53% 


53 


6 


Female 


34% 


34% 


38% 


40% 


40 


6 


Male 


35% 


35% 


39% 


41% 


41 


6 



Source: California Department of Education. 



The Relationship between State-level School Accountability 
and the Federal No Child Left Behind Act 

California’s accountability system is tied to the federal No Child Left Behind Act of 2001 (NCLB), 
which is being considered for reauthorization by Congress. The law enacted a nationwide, standards- 
based education reform program. The expectation was that setting high performance goals would be the 
most effective way to ensure success for all students. NCLB requires that all public schools of the same 
type (e.g., elementary, middle, and senior high schools) meet the same academic targets throughout any 
given state. Federal results are reported in terms of how well schools meet state-established annual 
growth targets. Under NCLB, schools that fall short of their growth targets are required to provide stu- 
dents with the option of transferring to a higher performance school. This option is referred to as public 
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school choice. By the academic year 2013-14, 100% of the nation’s public students are expected to be 
proficient in English/language arts and mathematics. States that elect to ignore certain provisions of 
NCLB are at risk of losing federal education funding. 

It is very unlikely that Congress will re-authorize NCLB without substantial revisions, given the level of 
criticism that has been directed at various provisions of the law since its enactment. The bipartisan Na- 
tional Conference of State Legislatures is requesting that Congress and the Bush Administration adjust 
the law in a manner that will enable states to have greater flexibility in responding to unique schooling 
needs. 



Schooling 

Schooling is the process of imparting knowledge and skills to individuals through cur- 
riculum and instruction, experiential learning, and work-based learning. Effective school- 
ing provides individuals with the necessary tools to become productive citizens, pursue 
higher education and lifelong learning, engage in meaningful employment, and work to- 
ward achieving their life goals. 



Next Steps 

hollowing Commission adoption of this report, staff will engage in three related planning efforts. Lirst, 
a follow-up agenda item will be prepared that summarizes the literature on valid research methods that 
have been used to assess the impact of leadership on school reform and student achievement. Second, 
Commission staff will continue to assist and advise the State University on the development of its edu- 
cational leadership proposals. Third, a working group comprised of Commissioners and representatives 
from the public and independent higher education systems, the Department of Education, the Depart- 
ment of Linance, and the Legislative Analyst Office will be formed to develop plans for implementing 
the recommendations contained in this report. 
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